test(a11y): e2e baseline tests for a11y screen reader descriptions#2684
Merged
Conversation
walterra
changed the title
walterra
changed the title
markov00
approved these changes
Jul 17, 2025
Collaborator
markov00
left a comment
markov00
left a comment
There was a problem hiding this comment.
Looks good to me, I just like to wait for @nickofthyme review on the CI part but looks working/running fine!
Contributor
Author
|
@nickofthyme as discussed I renamed the existing playwright tests/reports to include a reference to VRT so we have a better VRT/A11Y distinction for the e2e tests. |
nickofthyme
requested changes
Aug 5, 2025
Collaborator
nickofthyme
left a comment
nickofthyme
left a comment
There was a problem hiding this comment.
First pass on test code, looking at ci changes now.
nickofthyme
approved these changes
Aug 5, 2025
6 tasks
Closed
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds e2e accessibility tests to assert screen reader summaries for different chart types and configurations. The aim is to get a baseline of the current state of these summaries before we merge the improvements being worked on in #2683. This will allow us to track regressions/improvements in a better way.
Details
The accessibility e2e tests pick up the existing setup to run visual regression tests. Originally I thought we could run them as part of the visual regression tests, but then it would be tricky to run them locally because the visual ones don't pass there. So I implemented this via a separate command you can run via
yarn test:e2e:a11y. Like the visual regression tests, the a11y ones also walk through the storybook examples and then assert the screen reader description (what's within.echScreenReaderOnly).I added the tests to the CI setup too, the scripts are mostly copies of the ones used for the visual regression tests.
Open Question: We now run
yarn test:e2efor visual regression tests andyarn test:e2e:a11y. Should we be more explicit about the visual regression test ones (likeyarn test:e2e:vrt) - and also update descriptive texts to include VRT or Visual Regression Testing for the default e2e tests?Issues
Part of #36386.
Checklist
:interactions,:axis)closes #123,fixes #123)